from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.886	0.058	0.0	0.018	-1	1	0.663	1.915	0.046	9.338	0.227	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.886	0.058	0.0	0.018	-1	1	0.663	0.024	0.002	736.749	61.674	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.325	0.010	0.0	0.325	-1	1	1.000	1.915	0.046	0.170	0.007	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.325	0.010	0.0	0.325	-1	1	1.000	0.024	0.002	13.391	1.198	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.711	0.207	0.0	0.019	-1	5	0.757	2.804	0.063	6.673	0.167	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.711	0.207	0.0	0.019	-1	5	0.757	0.024	0.003	766.226	81.001	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.316	0.018	0.0	0.316	-1	5	1.000	2.804	0.063	0.113	0.007	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.316	0.018	0.0	0.316	-1	5	1.000	0.024	0.003	12.928	1.548	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.650	0.193	0.0	0.018	1	100	0.882	2.119	0.060	8.329	0.254	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.650	0.193	0.0	0.018	1	100	0.882	0.022	0.001	814.291	25.322	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.318	0.015	0.0	0.318	1	100	1.000	2.119	0.060	0.150	0.008	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.318	0.015	0.0	0.318	1	100	1.000	0.022	0.001	14.659	0.818	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.033	0.137	0.0	0.018	-1	100	0.882	2.832	0.034	6.367	0.091	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	18.033	0.137	0.0	0.018	-1	100	0.882	0.026	0.003	703.529	88.464	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.329	0.012	0.0	0.329	-1	100	1.000	2.832	0.034	0.116	0.004	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.329	0.012	0.0	0.329	-1	100	1.000	0.026	0.003	12.827	1.673	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.589	0.192	0.0	0.018	1	5	0.757	2.094	0.043	8.401	0.196	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.589	0.192	0.0	0.018	1	5	0.757	0.021	0.000	826.639	19.982	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.012	0.0	0.301	1	5	1.000	2.094	0.043	0.144	0.007	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.012	0.0	0.301	1	5	1.000	0.021	0.000	14.147	0.653	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.946	0.026	0.0	0.018	1	1	0.663	1.186	0.024	15.136	0.309	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.946	0.026	0.0	0.018	1	1	0.663	0.020	0.000	902.598	21.534	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.305	0.009	0.0	0.305	1	1	1.000	1.186	0.024	0.258	0.009	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.305	0.009	0.0	0.305	1	1	1.000	0.020	0.000	15.362	0.585	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.836	0.051	0.0	0.004	-1	1	0.896	1.724	0.034	2.225	0.053	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.836	0.051	0.0	0.004	-1	1	0.896	0.008	0.003	457.363	170.891	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.267	0.010	0.0	0.267	-1	1	1.000	1.724	0.034	0.155	0.006	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.267	0.010	0.0	0.267	-1	1	1.000	0.008	0.003	31.839	11.944	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.769	0.110	0.0	0.004	-1	5	0.922	2.731	0.075	1.380	0.055	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.769	0.110	0.0	0.004	-1	5	0.922	0.009	0.003	405.781	115.866	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.253	0.005	0.0	0.253	-1	5	1.000	2.731	0.075	0.093	0.003	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.253	0.005	0.0	0.253	-1	5	1.000	0.009	0.003	27.209	7.751	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.800	0.095	0.0	0.004	1	100	0.929	2.009	0.037	1.891	0.059	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.800	0.095	0.0	0.004	1	100	0.929	0.003	0.000	1182.809	144.220	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.248	0.011	0.0	0.248	1	100	1.000	2.009	0.037	0.124	0.006	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.248	0.011	0.0	0.248	1	100	1.000	0.003	0.000	77.245	9.831	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.885	0.093	0.0	0.004	-1	100	0.929	2.681	0.065	1.449	0.049	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.885	0.093	0.0	0.004	-1	100	0.929	0.010	0.003	407.092	117.140	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.244	0.009	0.0	0.244	-1	100	1.000	2.681	0.065	0.091	0.004	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.244	0.009	0.0	0.244	-1	100	1.000	0.010	0.003	25.594	7.405	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.716	0.108	0.0	0.004	1	5	0.922	1.971	0.042	1.885	0.068	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.716	0.108	0.0	0.004	1	5	0.922	0.003	0.001	1153.929	189.006	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.246	0.010	0.0	0.246	1	5	1.000	1.971	0.042	0.125	0.006	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.246	0.010	0.0	0.246	1	5	1.000	0.003	0.001	76.515	12.741	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.844	0.100	0.0	0.004	1	1	0.896	1.055	0.016	3.643	0.109	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.844	0.100	0.0	0.004	1	1	0.896	0.002	0.000	2101.284	332.122	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.247	0.016	0.0	0.247	1	1	1.000	1.055	0.016	0.234	0.015	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.247	0.016	0.0	0.247	1	1	1.000	0.002	0.000	134.831	22.707	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.649	0.000	0.0	0.130	-1	1	0.929	0.532	0.053	243.869	24.416	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	129.649	0.000	0.0	0.130	-1	1	0.929	0.004	0.002	31425.284	16377.294	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.706	0.184	0.0	2.706	-1	1	1.000	0.532	0.053	5.090	0.616	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.706	0.184	0.0	2.706	-1	1	1.000	0.004	0.002	655.851	344.693	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.282	0.000	0.0	0.130	-1	5	0.946	0.908	0.026	143.503	4.058	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	130.282	0.000	0.0	0.130	-1	5	0.946	0.004	0.001	33929.399	6346.377	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.804	0.110	0.0	2.804	-1	5	1.000	0.908	0.026	3.089	0.150	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.804	0.110	0.0	2.804	-1	5	1.000	0.004	0.001	730.300	139.593	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	133.110	0.000	0.0	0.133	1	100	0.951	5.269	0.155	25.262	0.744	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	133.110	0.000	0.0	0.133	1	100	0.951	0.003	0.001	39048.810	10697.238	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.658	0.087	0.0	2.658	1	100	1.000	5.269	0.155	0.504	0.022	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.658	0.087	0.0	2.658	1	100	1.000	0.003	0.001	779.604	215.091	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.252	0.000	0.0	0.131	-1	100	0.951	2.837	0.055	46.268	0.905	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	131.252	0.000	0.0	0.131	-1	100	0.951	0.005	0.002	24923.098	7891.429	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.692	0.137	0.0	2.692	-1	100	1.000	2.837	0.055	0.949	0.052	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.692	0.137	0.0	2.692	-1	100	1.000	0.005	0.002	511.223	163.945	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.236	0.000	0.0	0.136	1	5	0.946	1.536	0.031	88.724	1.785	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.236	0.000	0.0	0.136	1	5	0.946	0.002	0.000	81119.531	19171.322	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.847	0.157	0.0	2.847	1	5	1.000	1.536	0.031	1.854	0.109	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.847	0.157	0.0	2.847	1	5	1.000	0.002	0.000	1695.365	411.367	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	133.334	0.000	0.0	0.133	1	1	0.929	0.817	0.015	163.291	3.093	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	133.334	0.000	0.0	0.133	1	1	0.929	0.001	0.000	126930.413	44266.379	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.769	0.094	0.0	2.769	1	1	1.000	0.817	0.015	3.391	0.132	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.769	0.094	0.0	2.769	1	1	1.000	0.001	0.000	2635.827	923.590	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.004	0.0	0.000	-1	1	0.891	0.025	0.002	1.657	0.205	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.004	0.0	0.000	-1	1	0.891	0.002	0.001	16.749	4.082	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	-1	1	1.000	0.025	0.002	0.215	0.025	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	-1	1	1.000	0.002	0.001	2.171	0.522	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.003	0.0	0.000	-1	5	0.911	0.023	0.002	1.744	0.185	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.003	0.0	0.000	-1	5	0.911	0.002	0.001	15.869	4.319	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.023	0.002	0.228	0.025	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	-1	5	1.000	0.002	0.001	2.072	0.567	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.058	0.002	0.0	0.000	1	100	0.894	0.036	0.002	1.606	0.111	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.058	0.002	0.0	0.000	1	100	0.894	0.001	0.000	85.837	30.876	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.036	0.002	0.134	0.014	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	100	1.000	0.001	0.000	7.183	2.640	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.003	0.0	0.000	-1	100	0.894	0.037	0.006	1.628	0.289	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.060	0.003	0.0	0.000	-1	100	0.894	0.003	0.001	23.067	5.374	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	-1	100	1.000	0.037	0.006	0.148	0.030	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.001	0.0	0.005	-1	100	1.000	0.003	0.001	2.095	0.525	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.002	0.0	0.000	1	5	0.911	0.021	0.001	1.808	0.149	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.002	0.0	0.000	1	5	0.911	0.001	0.000	60.149	17.532	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.021	0.001	0.243	0.023	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.098	2.392	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.003	0.0	0.000	1	1	0.891	0.020	0.001	1.952	0.159	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.039	0.003	0.0	0.000	1	1	0.891	0.001	0.000	56.012	19.319	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.020	0.001	0.246	0.022	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	1	1.000	0.001	0.000	7.065	2.450	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	n_iter	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.473	0.038	300	0.002	0.000	0.824	0.120	0.008	3.929	0.407	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	0.473	0.038	300	0.002	0.000	0.824	0.017	0.001	27.043	2.469	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	0.373	0.010	300	0.000	0.373	1.000	0.120	0.008	3.097	0.216	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	0.373	0.010	300	0.000	0.373	1.000	0.017	0.001	21.317	1.063	See	See